Skip to content

viz: let apps choose the XR reference space - #899

Open
jiwenc-nv wants to merge 1 commit into
mainfrom
jiwenc/viz-xr-reference-space
Open

viz: let apps choose the XR reference space#899
jiwenc-nv wants to merge 1 commit into
mainfrom
jiwenc/viz-xr-reference-space

Conversation

@jiwenc-nv

@jiwenc-nv jiwenc-nv commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Description

VizSession never set a reference space, so OpenXrSession fell through to its XR_REFERENCE_SPACE_TYPE_LOCAL default — an origin at head height. Any app that draws world-locked geometry at a known height above the floor gets that height wrong by roughly a whole person, and the symptom is only visible on a headset. VizSessionConfig::xr_reference_space now names the choice (kLocal / kLocalFloor / kStage), exposed to Python as viz.XrReferenceSpace. The default stays kLocal, so no existing consumer changes behaviour.

An unavailable space throws naming the space rather than silently substituting a different origin, and create_reference_space logs what the runtime does offer beside what was asked for — the evidence anyone would want before adding a fallback chain.

The CloudXR WebXR client has its own half of this problem: auto prefers local-floor while a −155 cm vertical offset is applied whichever space it lands on, so the two corrections stack. The root README records the pairing that works; #871 asks why it is not the default.

Extracted from jiwenc/mujoco-xr-app. #900 (examples/mujoco_xr) is the first consumer and stacks on this.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Testing

Ubuntu 22.04 / aarch64 (Tegra). Full cmake --preset py3.12 -DBUILD_VIZ=ON build green; ctest -R viz green (4 viz_python_* + 3 camera_viz_*). Verified from Python that viz.XrReferenceSpace imports and VizSessionConfig().xr_reference_space defaults to kLocal.

kLocalFloor itself needs a headset and a CloudXR runtime, so the space actually being created is not exercised here.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the linter and formatter with SKIP=check-copyright-year pre-commit run --all-files
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix/feature works (or explained why not) — viz has no test that opens an XR session; the default-unchanged half is what the existing suite covers.
  • I have signed off all my commits (git commit -s) per the DCO

Summary by CodeRabbit

  • New Features
    • Added configurable OpenXR reference spaces: local, local-floor, and stage.
    • Exposed reference-space selection through Python session configuration.
    • Sessions now use the selected world-origin convention, with clear handling when it is unavailable.
  • Documentation
    • Added Quick Start guidance for keeping CloudXR WebXR content floor-anchored.
    • Documented local-floor configuration, zero vertical offset, defaults, and fallback behavior.
  • Diagnostics
    • Improved visibility into reference spaces supported by the XR runtime.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 38663830-70f7-4b9e-8611-42891c691296

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The change adds XrReferenceSpace with local, local-floor, and stage options. VizSession::Config now selects the reference space and defaults to local. Python bindings expose the enum and configuration property. Session setup converts the selected value to OpenXR. Runtime creation logs supported reference spaces. The README documents CloudXR WebXR settings for floor-anchored content.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant VizSessionConfig
  participant VizSession
  participant OpenXRSession
  VizSessionConfig->>VizSession: provide xr_reference_space
  VizSession->>OpenXRSession: set OpenXR reference_space_type
  OpenXRSession->>OpenXRSession: enumerate supported reference spaces
  OpenXRSession->>OpenXRSession: create requested reference space
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: configurable XR reference-space selection for applications.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jiwenc/viz-xr-reference-space

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/viz/xr/cpp/openxr_session.cpp (1)

361-362: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Include the requested reference space in the creation error.

check_xr receives only "xrCreateReferenceSpace". If creation fails, the exception does not name the requested space. The preceding log is conditional on successful enumeration.

Proposed fix
+    const std::string create_label =
+        std::string("xrCreateReferenceSpace(") + reference_space_name(type) + ")";
     XrSpace raw_ref = XR_NULL_HANDLE;
-    check_xr(xrCreateReferenceSpace(session_.get(), &info, &raw_ref), "xrCreateReferenceSpace");
+    check_xr(xrCreateReferenceSpace(session_.get(), &info, &raw_ref), create_label.c_str());
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/viz/xr/cpp/openxr_session.cpp` around lines 361 - 362, Update the
xrCreateReferenceSpace error context in the session initialization flow to
include the requested reference-space identifier alongside the operation name.
Modify the check_xr call after raw_ref is initialized, preserving the existing
behavior while ensuring failures identify which reference space was requested.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/viz/xr/cpp/openxr_session.cpp`:
- Around line 361-362: Update the xrCreateReferenceSpace error context in the
session initialization flow to include the requested reference-space identifier
alongside the operation name. Modify the check_xr call after raw_ref is
initialized, preserving the existing behavior while ensuring failures identify
which reference space was requested.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 344fddce-2d9a-4159-9550-8f437cafd8ec

📥 Commits

Reviewing files that changed from the base of the PR and between 26cab2d and e2dc74c.

📒 Files selected for processing (9)
  • README.md
  • src/python/isaacteleop/viz/__init__.py
  • src/viz/python/core_bindings.cpp
  • src/viz/python/session_bindings.cpp
  • src/viz/session/cpp/CMakeLists.txt
  • src/viz/session/cpp/inc/viz/session/viz_session.hpp
  • src/viz/session/cpp/inc/viz/session/xr_reference_space.hpp
  • src/viz/session/cpp/viz_session.cpp
  • src/viz/xr/cpp/openxr_session.cpp

VizSession never set a reference space, so OpenXrSession fell through to its
XR_REFERENCE_SPACE_TYPE_LOCAL default -- an origin at head height. Any app
drawing world-locked geometry at a known height above the floor gets that
height wrong by roughly a whole person, and the symptom is only visible on a
headset.

VizSessionConfig::xr_reference_space now names the choice (kLocal /
kLocalFloor / kStage), exposed to Python as viz.XrReferenceSpace. The default
stays kLocal, so no existing consumer changes behaviour. An unavailable space
throws naming the space rather than silently substituting a different origin,
and create_reference_space logs what the runtime does offer beside what was
asked for.

The CloudXR WebXR client has the other half of this problem: `auto` prefers
local-floor while a -155 cm vertical offset is applied whichever space it
lands on, so the two corrections stack. The README records the pairing that
works; #871 asks why it is not the default.

Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
@jiwenc-nv
jiwenc-nv force-pushed the jiwenc/viz-xr-reference-space branch from e2dc74c to 0bc685c Compare August 5, 2026 16:30
@jiwenc-nv
jiwenc-nv requested a review from farbod-nv August 5, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant